最近已经提出了压缩的随机梯度下降（SGD）算法，以解决分布式和分散的优化问题（例如在联合机器学习中出现的问题）中的通信瓶颈。现有的压缩SGD算法假定使用非自适应的阶梯尺寸（恒定或减小）来提供理论收敛保证。通常，在实践中对数据集和学习算法进行微调，以提供良好的经验性能。在许多学习方案中，这种微调可能是不切实际的，因此，使用自适应阶梯尺寸研究压缩SGD是很感兴趣的。由SGD在未压缩环境中有效训练神经网络的自适应阶梯尺寸方法的先前工作的激励，我们为压缩SGD开发了一种自适应阶梯尺寸方法。特别是，我们在压缩SGD中引入了一种缩放技术，我们用来在插值条件下为凸 - 平滑和强凸 - 平滑目标建立订单 - 最佳收敛速率，并在强烈的增长下为健康）状况。我们还通过仿真示例显示，如果没有这种缩放，算法就无法收敛。我们介绍了现实世界数据集的深神经网络的实验结果，并将我们提出的算法的性能与先前提出的文献压缩SGD方法进行比较，并在Resnet-18，Resnet-34和Densenet架构上的CIFAR-100架构上的性能提高了和CIFAR-10数据集的各种压缩级别。

我们研究了分布外（OOD）检测的问题，也就是说，检测学习算法的输出是否可以在推理时间得到信任。尽管已经在先前的工作中提出了许多OOD检测的测试，但缺乏研究此问题的正式框架。我们提出了一个关于OOD概念的定义，其中包括输入分布和学习算法，该算法为构建强大的OOD检测测试提供了见解。我们提出了一个多个假设测试的启发程序，以系统地结合学习算法的任何数量的不同统计数据，使用保形p值。我们进一步为将分配样本分类为OOD的概率提供了强有力的保证。在我们的实验中，我们发现在先前工作中提出的基于阈值的测试在特定的设置中表现良好，但在不同类型的OOD实例中并不均匀。相比之下，我们提出的方法结合了多个统计数据在不同的数据集和神经网络中表现出色。

我们考虑一个完全分散的多人多手随机多武装匪盗匪徒，其中玩家不能互相通信，并且只能观察自己的行为和奖励。环境可能与不同的播放器不同，$ \ texit {i.e.} $，给定臂的奖励分布在球员之间是异构的。在碰撞的情况下（当多个玩家播放相同的手臂时），我们允许碰撞玩家接收非零奖励。播放武器的时间 - 地平线$ t $是\ emph {否}对玩家已知。在此设置中，允许玩家的数量大于武器的数量，我们展示了一项达到订单优化预期令人遗憾的政策$ O（\ log ^ {1 + delta} t）$有些$ 0 <\ delta <1 $超过时间的时间$ t $。IEEE关于信息理论的交易中接受了本文。

In this paper we explore the task of modeling (semi) structured object sequences; in particular we focus our attention on the problem of developing a structure-aware input representation for such sequences. In such sequences, we assume that each structured object is represented by a set of key-value pairs which encode the attributes of the structured object. Given a universe of keys, a sequence of structured objects can then be viewed as an evolution of the values for each key, over time. We encode and construct a sequential representation using the values for a particular key (Temporal Value Modeling - TVM) and then self-attend over the set of key-conditioned value sequences to a create a representation of the structured object sequence (Key Aggregation - KA). We pre-train and fine-tune the two components independently and present an innovative training schedule that interleaves the training of both modules with shared attention heads. We find that this iterative two part-training results in better performance than a unified network with hierarchical encoding as well as over, other methods that use a {\em record-view} representation of the sequence \cite{de2021transformers4rec} or a simple {\em flattened} representation of the sequence. We conduct experiments using real-world data to demonstrate the advantage of interleaving TVM-KA on multiple tasks and detailed ablation studies motivating our modeling choices. We find that our approach performs better than flattening sequence objects and also allows us to operate on significantly larger sequences than existing methods.

Optical coherence tomography (OCT) captures cross-sectional data and is used for the screening, monitoring, and treatment planning of retinal diseases. Technological developments to increase the speed of acquisition often results in systems with a narrower spectral bandwidth, and hence a lower axial resolution. Traditionally, image-processing-based techniques have been utilized to reconstruct subsampled OCT data and more recently, deep-learning-based methods have been explored. In this study, we simulate reduced axial scan (A-scan) resolution by Gaussian windowing in the spectral domain and investigate the use of a learning-based approach for image feature reconstruction. In anticipation of the reduced resolution that accompanies wide-field OCT systems, we build upon super-resolution techniques to explore methods to better aid clinicians in their decision-making to improve patient outcomes, by reconstructing lost features using a pixel-to-pixel approach with an altered super-resolution generative adversarial network (SRGAN) architecture.

Real-life tools for decision-making in many critical domains are based on ranking results. With the increasing awareness of algorithmic fairness, recent works have presented measures for fairness in ranking. Many of those definitions consider the representation of different ``protected groups'', in the top-$k$ ranked items, for any reasonable $k$. Given the protected groups, confirming algorithmic fairness is a simple task. However, the groups' definitions may be unknown in advance. In this paper, we study the problem of detecting groups with biased representation in the top-$k$ ranked items, eliminating the need to pre-define protected groups. The number of such groups possible can be exponential, making the problem hard. We propose efficient search algorithms for two different fairness measures: global representation bounds, and proportional representation. Then we propose a method to explain the bias in the representations of groups utilizing the notion of Shapley values. We conclude with an experimental study, showing the scalability of our approach and demonstrating the usefulness of the proposed algorithms.

The previous fine-grained datasets mainly focus on classification and are often captured in a controlled setup, with the camera focusing on the objects. We introduce the first Fine-Grained Vehicle Detection (FGVD) dataset in the wild, captured from a moving camera mounted on a car. It contains 5502 scene images with 210 unique fine-grained labels of multiple vehicle types organized in a three-level hierarchy. While previous classification datasets also include makes for different kinds of cars, the FGVD dataset introduces new class labels for categorizing two-wheelers, autorickshaws, and trucks. The FGVD dataset is challenging as it has vehicles in complex traffic scenarios with intra-class and inter-class variations in types, scale, pose, occlusion, and lighting conditions. The current object detectors like yolov5 and faster RCNN perform poorly on our dataset due to a lack of hierarchical modeling. Along with providing baseline results for existing object detectors on FGVD Dataset, we also present the results of a combination of an existing detector and the recent Hierarchical Residual Network (HRN) classifier for the FGVD task. Finally, we show that FGVD vehicle images are the most challenging to classify among the fine-grained datasets.

Three main points: 1. Data Science (DS) will be increasingly important to heliophysics; 2. Methods of heliophysics science discovery will continually evolve, requiring the use of learning technologies [e.g., machine learning (ML)] that are applied rigorously and that are capable of supporting discovery; and 3. To grow with the pace of data, technology, and workforce changes, heliophysics requires a new approach to the representation of knowledge.

In the Earth's magnetosphere, there are fewer than a dozen dedicated probes beyond low-Earth orbit making in-situ observations at any given time. As a result, we poorly understand its global structure and evolution, the mechanisms of its main activity processes, magnetic storms, and substorms. New Artificial Intelligence (AI) methods, including machine learning, data mining, and data assimilation, as well as new AI-enabled missions will need to be developed to meet this Sparse Data challenge.

Dataset scaling, also known as normalization, is an essential preprocessing step in a machine learning pipeline. It is aimed at adjusting attributes scales in a way that they all vary within the same range. This transformation is known to improve the performance of classification models, but there are several scaling techniques to choose from, and this choice is not generally done carefully. In this paper, we execute a broad experiment comparing the impact of 5 scaling techniques on the performances of 20 classification algorithms among monolithic and ensemble models, applying them to 82 publicly available datasets with varying imbalance ratios. Results show that the choice of scaling technique matters for classification performance, and the performance difference between the best and the worst scaling technique is relevant and statistically significant in most cases. They also indicate that choosing an inadequate technique can be more detrimental to classification performance than not scaling the data at all. We also show how the performance variation of an ensemble model, considering different scaling techniques, tends to be dictated by that of its base model. Finally, we discuss the relationship between a model's sensitivity to the choice of scaling technique and its performance and provide insights into its applicability on different model deployment scenarios. Full results and source code for the experiments in this paper are available in a GitHub repository.\footnote{https://github.com/amorimlb/scaling\_matters}